{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "validation.ipynb",
      "version": "0.3.2",
      "views": {},
      "default_view": {},
      "provenance": [],
      "collapsed_sections": [
        "4Xp9NhOCYSuz",
        "pECTKgw5ZvFK",
        "dER2_43pWj1T",
        "I-La4N9ObC1x",
        "yTghc_5HkJDW",
        "copyright-notice"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "source": [
        "#### Copyright 2017 Google LLC."
      ],
      "cell_type": "markdown",
      "metadata": {
        "id": "copyright-notice",
        "colab_type": "text"
      }
    },
    {
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "outputs": [],
      "execution_count": 0,
      "cell_type": "code",
      "metadata": {
        "cellView": "both",
        "id": "copyright-notice2",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "wait_interval": 0,
            "startup": false
          }
        }
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zbIgBK-oXHO7",
        "colab_type": "text"
      },
      "source": [
        " # Validation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WNX0VyBpHpCX",
        "colab_type": "text"
      },
      "source": [
        " **Objectifs d'apprentissage\u00a0:**\n",
        "  * Utiliser plusieurs caract\u00e9ristiques, plut\u00f4t qu'une seule, pour am\u00e9liorer encore l'efficacit\u00e9 d'un mod\u00e8le\n",
        "  * D\u00e9boguer les erreurs au niveau des donn\u00e9es d'entr\u00e9e du mod\u00e8le\n",
        "  * Utiliser un ensemble de donn\u00e9es d'\u00e9valuation pour d\u00e9terminer si un mod\u00e8le surapprend les donn\u00e9es de validation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "za0m1T8CHpCY",
        "colab_type": "text"
      },
      "source": [
        " Comme pour les exercices pr\u00e9c\u00e9dents, vous allez utiliser l'ensemble de donn\u00e9es sur l'immobilier en Californie pour t\u00e2cher de pr\u00e9dire la valeur m\u00e9diane des logements (`median_house_value`) au niveau de l'\u00eelot urbain, sur la base du recensement de 1990."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r2zgMfWDWF12",
        "colab_type": "text"
      },
      "source": [
        " ## Configuration"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8jErhkLzWI1B",
        "colab_type": "text"
      },
      "source": [
        " Commencez par charger et pr\u00e9parer les donn\u00e9es. Cette fois, vous allez utiliser plusieurs caract\u00e9ristiques. Vous allez donc modulariser la logique pour pr\u00e9traiter un peu les caract\u00e9ristiques\u00a0:"
      ]
    },
    {
      "metadata": {
        "id": "PwS5Bhm6HpCZ",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "from __future__ import print_function\n",
        "\n",
        "import math\n",
        "\n",
        "from IPython import display\n",
        "from matplotlib import cm\n",
        "from matplotlib import gridspec\n",
        "from matplotlib import pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn import metrics\n",
        "import tensorflow as tf\n",
        "from tensorflow.python.data import Dataset\n",
        "\n",
        "tf.logging.set_verbosity(tf.logging.ERROR)\n",
        "pd.options.display.max_rows = 10\n",
        "pd.options.display.float_format = '{:.1f}'.format\n",
        "\n",
        "california_housing_dataframe = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\", sep=\",\")\n",
        "\n",
        "# california_housing_dataframe = california_housing_dataframe.reindex(\n",
        "#     np.random.permutation(california_housing_dataframe.index))"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "J2ZyTzX0HpCc",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "def preprocess_features(california_housing_dataframe):\n",
        "  \"\"\"Prepares input features from California housing data set.\n",
        "\n",
        "  Args:\n",
        "    california_housing_dataframe: A Pandas DataFrame expected to contain data\n",
        "      from the California housing data set.\n",
        "  Returns:\n",
        "    A DataFrame that contains the features to be used for the model, including\n",
        "    synthetic features.\n",
        "  \"\"\"\n",
        "  selected_features = california_housing_dataframe[\n",
        "    [\"latitude\",\n",
        "     \"longitude\",\n",
        "     \"housing_median_age\",\n",
        "     \"total_rooms\",\n",
        "     \"total_bedrooms\",\n",
        "     \"population\",\n",
        "     \"households\",\n",
        "     \"median_income\"]]\n",
        "  processed_features = selected_features.copy()\n",
        "  # Create a synthetic feature.\n",
        "  processed_features[\"rooms_per_person\"] = (\n",
        "    california_housing_dataframe[\"total_rooms\"] /\n",
        "    california_housing_dataframe[\"population\"])\n",
        "  return processed_features\n",
        "\n",
        "def preprocess_targets(california_housing_dataframe):\n",
        "  \"\"\"Prepares target features (i.e., labels) from California housing data set.\n",
        "\n",
        "  Args:\n",
        "    california_housing_dataframe: A Pandas DataFrame expected to contain data\n",
        "      from the California housing data set.\n",
        "  Returns:\n",
        "    A DataFrame that contains the target feature.\n",
        "  \"\"\"\n",
        "  output_targets = pd.DataFrame()\n",
        "  # Scale the target to be in units of thousands of dollars.\n",
        "  output_targets[\"median_house_value\"] = (\n",
        "    california_housing_dataframe[\"median_house_value\"] / 1000.0)\n",
        "  return output_targets"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sZSIaDiaHpCf",
        "colab_type": "text"
      },
      "source": [
        " Pour l'**ensemble d'apprentissage**, vous allez choisir les 12\u00a0000\u00a0premiers exemples, sur un total de 17\u00a0000."
      ]
    },
    {
      "metadata": {
        "id": "P9wejvw7HpCf",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "training_examples = preprocess_features(california_housing_dataframe.head(12000))\n",
        "training_examples.describe()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "JlkgPR-SHpCh",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "training_targets = preprocess_targets(california_housing_dataframe.head(12000))\n",
        "training_targets.describe()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5l1aA2xOHpCj",
        "colab_type": "text"
      },
      "source": [
        " Pour l'**ensemble de validation**, vous allez choisir les 5\u00a0000\u00a0derniers exemples, sur un total de 17\u00a0000."
      ]
    },
    {
      "metadata": {
        "id": "fLYXLWAiHpCk",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "validation_examples = preprocess_features(california_housing_dataframe.tail(5000))\n",
        "validation_examples.describe()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "oVPcIT3BHpCm",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "validation_targets = preprocess_targets(california_housing_dataframe.tail(5000))\n",
        "validation_targets.describe()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z3TZV1pgfZ1n",
        "colab_type": "text"
      },
      "source": [
        " ## T\u00e2che\u00a01\u00a0: Examiner les donn\u00e9es\n",
        "Examinons les donn\u00e9es ci-dessus. On voit que `9` caract\u00e9ristiques d'entr\u00e9e peuvent \u00eatre utilis\u00e9es.\n",
        "\n",
        "Jetez un rapide coup d'\u0153il sur le tableau des valeurs. Tout vous semble correct\u00a0? Voyez combien de probl\u00e8mes vous pouvez relever. Ne vous inqui\u00e9tez pas, aucune connaissance en statistiques n'est requise. Il suffit de faire preuve de bon sens.\n",
        "\n",
        "Apr\u00e8s avoir eu l'occasion d'examiner les donn\u00e9es par vous-m\u00eame, consultez la solution. Vous y trouverez d'autres id\u00e9es sur la mani\u00e8re de valider les donn\u00e9es."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Xp9NhOCYSuz",
        "colab_type": "text"
      },
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher la solution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gqeRmK57YWpy",
        "colab_type": "text"
      },
      "source": [
        " Voyons si les donn\u00e9es sont conformes \u00e0 quelques attentes \u00e9l\u00e9mentaires\u00a0:\n",
        "\n",
        "* Pour certaines valeurs, telles que `median_house_value`, vous pouvez v\u00e9rifier si les plages sont acceptables (pour rappel, ces donn\u00e9es datent de 1990\u00a0!).\n",
        "\n",
        "* Pour d'autres valeurs, comme `latitude` et `longitude`, vous pouvez v\u00e9rifier rapidement \u00e0 l'aide d'une recherche Google si elles sont conformes aux r\u00e9sultats que l'on s'attend \u00e0 obtenir.\n",
        "\n",
        "En y regardant de plus pr\u00e8s, on peut constater quelques anomalies\u00a0:\n",
        "\n",
        "* La valeur `median_income` se situe sur une \u00e9chelle allant d'environ 3 \u00e0 15. Rien ne permet de d\u00e9terminer avec certitude \u00e0 quoi cette \u00e9chelle fait r\u00e9f\u00e9rence\u00a0: peut-\u00eatre s'agit-il d'une \u00e9chelle logarithmique\u00a0? On ne trouve aucune information \u00e0 ce sujet\u00a0; on peut simplement supposer que les valeurs plus \u00e9lev\u00e9es correspondent \u00e0 un revenu sup\u00e9rieur.\n",
        "\n",
        "* La valeur `median_house_value` maximale est de 500\u00a0001. Cela ressemble \u00e0 une limite artificielle.\n",
        "\n",
        "* La caract\u00e9ristique `rooms_per_person` se trouve g\u00e9n\u00e9ralement sur une \u00e9chelle raisonnable, avec une valeur du 75e\u00a0percentile correspondant approximativement \u00e0 2. Il existe toutefois quelques valeurs particuli\u00e8rement grandes, comme 18 ou 55, qui peuvent indiquer une certaine corruption des donn\u00e9es.\n",
        "\n",
        "Pour l'instant, vous utiliserez ces caract\u00e9ristiques telles quelles. Cependant, on peut esp\u00e9rer que des exemples de ce type vous aident \u00e0 comprendre comment v\u00e9rifier les donn\u00e9es qui vous arrivent en provenance d'une source inconnue."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fXliy7FYZZRm",
        "colab_type": "text"
      },
      "source": [
        " ## T\u00e2che\u00a02\u00a0: Repr\u00e9senter graphiquement la latitude/longitude par rapport \u00e0 la valeur m\u00e9diane des logements"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aJIWKBdfsDjg",
        "colab_type": "text"
      },
      "source": [
        " Examinons deux caract\u00e9ristiques en particulier\u00a0: **`latitude`** et **`longitude`**. Il s'agit des coordonn\u00e9es g\u00e9ographiques de l'\u00eelot urbain \u00e9tudi\u00e9.\n",
        "\n",
        "Cela peut produire une belle visualisation. Vous allez repr\u00e9senter graphiquement `latitude` et `longitude`, puis utiliser des couleurs pour afficher la valeur m\u00e9diane des logements (`median_house_value`)."
      ]
    },
    {
      "metadata": {
        "id": "5_LD23bJ06TW",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "plt.figure(figsize=(13, 8))\n",
        "\n",
        "ax = plt.subplot(1, 2, 1)\n",
        "ax.set_title(\"Validation Data\")\n",
        "\n",
        "ax.set_autoscaley_on(False)\n",
        "ax.set_ylim([32, 43])\n",
        "ax.set_autoscalex_on(False)\n",
        "ax.set_xlim([-126, -112])\n",
        "plt.scatter(validation_examples[\"longitude\"],\n",
        "            validation_examples[\"latitude\"],\n",
        "            cmap=\"coolwarm\",\n",
        "            c=validation_targets[\"median_house_value\"] / validation_targets[\"median_house_value\"].max())\n",
        "\n",
        "ax = plt.subplot(1,2,2)\n",
        "ax.set_title(\"Training Data\")\n",
        "\n",
        "ax.set_autoscaley_on(False)\n",
        "ax.set_ylim([32, 43])\n",
        "ax.set_autoscalex_on(False)\n",
        "ax.set_xlim([-126, -112])\n",
        "plt.scatter(training_examples[\"longitude\"],\n",
        "            training_examples[\"latitude\"],\n",
        "            cmap=\"coolwarm\",\n",
        "            c=training_targets[\"median_house_value\"] / training_targets[\"median_house_value\"].max())\n",
        "_ = plt.plot()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "32_DbjnfXJlC",
        "colab_type": "text"
      },
      "source": [
        " Une seconde\u00a0! Nous aurions d\u00fb avoir une belle carte de l\u2019\u00c9tat de Californie, sur laquelle les villes les plus ch\u00e8res, comme San\u00a0Francisco et Los\u00a0Angeles, sont indiqu\u00e9es en rouge.\n",
        "\n",
        "C'est en quelque sorte ce que fait l'ensemble d'apprentissage, par rapport \u00e0 une [carte r\u00e9elle](https://www.google.com/maps/place/California/@37.1870174,-123.7642688,6z/data=!3m1!4b1!4m2!3m1!1s0x808fb9fe5f285e3d:0x8b5109a227086f55), mais ce n'est, de toute \u00e9vidence, pas le cas de l'ensemble de validation.\n",
        "\n",
        "**Revenez en arri\u00e8re et examinez \u00e0 nouveau les donn\u00e9es de la T\u00e2che\u00a01.**\n",
        "\n",
        "Voyez-vous d'autres diff\u00e9rences dans les r\u00e9partitions de caract\u00e9ristiques ou cibles entre les donn\u00e9es d'apprentissage et de validation\u00a0?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pECTKgw5ZvFK",
        "colab_type": "text"
      },
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher la solution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "49NC4_KIZxk_",
        "colab_type": "text"
      },
      "source": [
        " En observant les tableaux de statistiques r\u00e9capitulatives ci-dessus, on est droit de se demander comment quelqu'un pourrait bien effectuer une v\u00e9rification utile des donn\u00e9es. Quelle est la valeur correcte du 75<sup>e</sup> percentile pour le nombre total de pi\u00e8ces (total_rooms) par \u00eelot urbain\u00a0?\n",
        "\n",
        "Il est important de noter que, pour une caract\u00e9ristique ou une colonne donn\u00e9e, la r\u00e9partition des valeurs entre les ensembles d'apprentissage et de validation devrait \u00eatre sensiblement \u00e9gale.\n",
        "\n",
        "Le fait que cela ne soit pas le cas est vraiment pr\u00e9occupant, et indique une probable erreur dans la m\u00e9thode utilis\u00e9e pour cr\u00e9er les ensembles d'apprentissage et de validation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "025Ky0Dq9ig0",
        "colab_type": "text"
      },
      "source": [
        " ## T\u00e2che\u00a03\u00a0: Revenir au code d'importation et de pr\u00e9traitement des donn\u00e9es, et d\u00e9tecter d'\u00e9ventuels bugs\n",
        "Le cas \u00e9ch\u00e9ant, corrigez le bug. Ne passez pas plus d'une ou deux minutes \u00e0 rechercher le bug. Si vous ne le trouvez pas, v\u00e9rifiez la solution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JFsd2eWHAMdy",
        "colab_type": "text"
      },
      "source": [
        " Apr\u00e8s avoir d\u00e9tect\u00e9 et corrig\u00e9 l'erreur, ex\u00e9cutez \u00e0 nouveau la cellule de tra\u00e7age `latitude`/`longitude` ci-dessus et confirmez l'am\u00e9lioration des \u00e9valuations d'int\u00e9grit\u00e9.\n",
        "\n",
        "Au passage, signalons qu'il y a ici un enseignement important \u00e0 tirer.\n",
        "\n",
        "**Dans le domaine du ML, le d\u00e9bogage s'apparente davantage \u00e0 du *d\u00e9bogage de donn\u00e9es* qu'\u00e0 du d\u00e9bogage de code.**\n",
        "\n",
        "Si les donn\u00e9es sont erron\u00e9es, m\u00eame le code ML le plus sophistiqu\u00e9 ne peut pas faire de miracles."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dER2_43pWj1T",
        "colab_type": "text"
      },
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher la solution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BnEVbYJvW2wu",
        "colab_type": "text"
      },
      "source": [
        " Voyons \u00e0 pr\u00e9sent comment les donn\u00e9es sont rendues al\u00e9atoires lors de leur lecture.\n",
        "\n",
        "Si une application al\u00e9atoire des donn\u00e9es n'est pas effectu\u00e9e correctement avant de cr\u00e9er des ensembles d'apprentissage et de validation, des probl\u00e8mes peuvent survenir en cas d'obtention de donn\u00e9es class\u00e9es\u00a0; ce qui semble \u00eatre le cas ici."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xCdqLpQyAos2",
        "colab_type": "text"
      },
      "source": [
        " ## T\u00e2che\u00a04\u00a0: Entra\u00eener et \u00e9valuer un mod\u00e8le\n",
        "\n",
        "**Consacrez environ cinq\u00a0minutes \u00e0 exp\u00e9rimenter diff\u00e9rents hyperparam\u00e8tres. Essayez d'obtenir les meilleures performances de validation possibles.**\n",
        "\n",
        "Vous allez ensuite entra\u00eener une variable ind\u00e9pendante lin\u00e9aire \u00e0 l'aide de toutes les caract\u00e9ristiques dans l'ensemble de donn\u00e9es et voir quel est le r\u00e9sultat.\n",
        "\n",
        "D\u00e9finissez la m\u00eame fonction d'entr\u00e9e que celle utilis\u00e9e pr\u00e9c\u00e9demment pour charger les donn\u00e9es dans un mod\u00e8le TensorFlow.\n"
      ]
    },
    {
      "metadata": {
        "id": "rzcIPGxxgG0t",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):\n",
        "    \"\"\"Trains a linear regression model of multiple features.\n",
        "  \n",
        "    Args:\n",
        "      features: pandas DataFrame of features\n",
        "      targets: pandas DataFrame of targets\n",
        "      batch_size: Size of batches to be passed to the model\n",
        "      shuffle: True or False. Whether to shuffle the data.\n",
        "      num_epochs: Number of epochs for which data should be repeated. None = repeat indefinitely\n",
        "    Returns:\n",
        "      Tuple of (features, labels) for next data batch\n",
        "    \"\"\"\n",
        "    \n",
        "    # Convert pandas data into a dict of np arrays.\n",
        "    features = {key:np.array(value) for key,value in dict(features).items()}                                           \n",
        " \n",
        "    # Construct a dataset, and configure batching/repeating.\n",
        "    ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit\n",
        "    ds = ds.batch(batch_size).repeat(num_epochs)\n",
        "    \n",
        "    # Shuffle the data, if specified.\n",
        "    if shuffle:\n",
        "      ds = ds.shuffle(10000)\n",
        "    \n",
        "    # Return the next batch of data.\n",
        "    features, labels = ds.make_one_shot_iterator().get_next()\n",
        "    return features, labels"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CvrKoBmNgRCO",
        "colab_type": "text"
      },
      "source": [
        " \u00c9tant donn\u00e9 que vous travaillez \u00e0 pr\u00e9sent avec plusieurs caract\u00e9ristiques d'entr\u00e9e, vous allez modulariser le code pour configurer des colonnes de caract\u00e9ristiques dans une fonction distincte. Pour l'instant, ce code est relativement simple, car toutes les caract\u00e9ristiques sont num\u00e9riques. Cependant, nous allons le d\u00e9velopper \u00e0 mesure que nous utiliserons d'autres types de caract\u00e9ristiques dans les prochains exercices.)"
      ]
    },
    {
      "metadata": {
        "id": "wEW5_XYtgZ-H",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "def construct_feature_columns(input_features):\n",
        "  \"\"\"Construct the TensorFlow Feature Columns.\n",
        "\n",
        "  Args:\n",
        "    input_features: The names of the numerical input features to use.\n",
        "  Returns:\n",
        "    A set of feature columns\n",
        "  \"\"\" \n",
        "  return set([tf.feature_column.numeric_column(my_feature)\n",
        "              for my_feature in input_features])"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D0o2wnnzf8BD",
        "colab_type": "text"
      },
      "source": [
        " Vous allez maintenant ex\u00e9cuter le code `train_model()` ci-dessous pour configurer les fonctions d'entr\u00e9e et calculer des pr\u00e9dictions.\n",
        "\n",
        "**REMARQUE\u00a0:** Il est permis de faire r\u00e9f\u00e9rence au code des exercices pr\u00e9c\u00e9dents, mais veillez \u00e0 appeler `predict()` sur les ensembles de donn\u00e9es appropri\u00e9s.\n",
        "\n",
        "Comparez les co\u00fbts sur les donn\u00e9es d'apprentissage et de validation. Avec une seule caract\u00e9ristique brute, la meilleure valeur RMSE (racine carr\u00e9e de l'erreur quadratique moyenne) obtenue \u00e9tait d'environ\u00a0180.\n",
        "\n",
        "Voyons maintenant dans quelle mesure l'utilisation de plusieurs caract\u00e9ristiques peut \u00eatre b\u00e9n\u00e9fique.\n",
        "\n",
        "V\u00e9rifiez les donn\u00e9es en utilisant quelques-unes des m\u00e9thodes mentionn\u00e9es pr\u00e9c\u00e9demment, \u00e0 savoir\u00a0:\n",
        "\n",
        "   * Comparaison des r\u00e9partitions de pr\u00e9dictions et des valeurs cibles r\u00e9elles\n",
        "\n",
        "   * Cr\u00e9ation d'un diagramme de dispersion des pr\u00e9dictions par rapport aux valeurs cibles\n",
        "\n",
        "   * Cr\u00e9ation de deux diagrammes de dispersion des donn\u00e9es de validation \u00e0 l'aide de `latitude` et `longitude`\u00a0:\n",
        "      * Un diagramme associant une couleur \u00e0 la valeur `median_house_value` cible r\u00e9elle\n",
        "      * Un deuxi\u00e8me diagramme associant une couleur \u00e0 la valeur `median_house_value` pr\u00e9dite pour un affichage comparatif."
      ]
    },
    {
      "metadata": {
        "id": "UXt0_4ZTEf4V",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "def train_model(\n",
        "    learning_rate,\n",
        "    steps,\n",
        "    batch_size,\n",
        "    training_examples,\n",
        "    training_targets,\n",
        "    validation_examples,\n",
        "    validation_targets):\n",
        "  \"\"\"Trains a linear regression model of multiple features.\n",
        "  \n",
        "  In addition to training, this function also prints training progress information,\n",
        "  as well as a plot of the training and validation loss over time.\n",
        "  \n",
        "  Args:\n",
        "    learning_rate: A `float`, the learning rate.\n",
        "    steps: A non-zero `int`, the total number of training steps. A training step\n",
        "      consists of a forward and backward pass using a single batch.\n",
        "    batch_size: A non-zero `int`, the batch size.\n",
        "    training_examples: A `DataFrame` containing one or more columns from\n",
        "      `california_housing_dataframe` to use as input features for training.\n",
        "    training_targets: A `DataFrame` containing exactly one column from\n",
        "      `california_housing_dataframe` to use as target for training.\n",
        "    validation_examples: A `DataFrame` containing one or more columns from\n",
        "      `california_housing_dataframe` to use as input features for validation.\n",
        "    validation_targets: A `DataFrame` containing exactly one column from\n",
        "      `california_housing_dataframe` to use as target for validation.\n",
        "      \n",
        "  Returns:\n",
        "    A `LinearRegressor` object trained on the training data.\n",
        "  \"\"\"\n",
        "\n",
        "  periods = 10\n",
        "  steps_per_period = steps / periods\n",
        "  \n",
        "  # Create a linear regressor object.\n",
        "  my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n",
        "  my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n",
        "  linear_regressor = tf.estimator.LinearRegressor(\n",
        "      feature_columns=construct_feature_columns(training_examples),\n",
        "      optimizer=my_optimizer\n",
        "  )\n",
        "  \n",
        "  # 1. Create input functions.\n",
        "  training_input_fn = # YOUR CODE HERE\n",
        "  predict_training_input_fn = # YOUR CODE HERE\n",
        "  predict_validation_input_fn = # YOUR CODE HERE\n",
        "  \n",
        "  # Train the model, but do so inside a loop so that we can periodically assess\n",
        "  # loss metrics.\n",
        "  print(\"Training model...\")\n",
        "  print(\"RMSE (on training data):\")\n",
        "  training_rmse = []\n",
        "  validation_rmse = []\n",
        "  for period in range (0, periods):\n",
        "    # Train the model, starting from the prior state.\n",
        "    linear_regressor.train(\n",
        "        input_fn=training_input_fn,\n",
        "        steps=steps_per_period,\n",
        "    )\n",
        "    # 2. Take a break and compute predictions.\n",
        "    training_predictions = # YOUR CODE HERE\n",
        "    validation_predictions = # YOUR CODE HERE\n",
        "    \n",
        "    # Compute training and validation loss.\n",
        "    training_root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(training_predictions, training_targets))\n",
        "    validation_root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(validation_predictions, validation_targets))\n",
        "    # Occasionally print the current loss.\n",
        "    print(\"  period %02d : %0.2f\" % (period, training_root_mean_squared_error))\n",
        "    # Add the loss metrics from this period to our list.\n",
        "    training_rmse.append(training_root_mean_squared_error)\n",
        "    validation_rmse.append(validation_root_mean_squared_error)\n",
        "  print(\"Model training finished.\")\n",
        "\n",
        "  # Output a graph of loss metrics over periods.\n",
        "  plt.ylabel(\"RMSE\")\n",
        "  plt.xlabel(\"Periods\")\n",
        "  plt.title(\"Root Mean Squared Error vs. Periods\")\n",
        "  plt.tight_layout()\n",
        "  plt.plot(training_rmse, label=\"training\")\n",
        "  plt.plot(validation_rmse, label=\"validation\")\n",
        "  plt.legend()\n",
        "\n",
        "  return linear_regressor"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "zFFRmvUGh8wd",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "linear_regressor = train_model(\n",
        "    # TWEAK THESE VALUES TO SEE HOW MUCH YOU CAN IMPROVE THE RMSE\n",
        "    learning_rate=0.00001,\n",
        "    steps=100,\n",
        "    batch_size=1,\n",
        "    training_examples=training_examples,\n",
        "    training_targets=training_targets,\n",
        "    validation_examples=validation_examples,\n",
        "    validation_targets=validation_targets)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I-La4N9ObC1x",
        "colab_type": "text"
      },
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher une solution."
      ]
    },
    {
      "metadata": {
        "id": "Xyz6n1YHbGef",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "def train_model(\n",
        "    learning_rate,\n",
        "    steps,\n",
        "    batch_size,\n",
        "    training_examples,\n",
        "    training_targets,\n",
        "    validation_examples,\n",
        "    validation_targets):\n",
        "  \"\"\"Trains a linear regression model of multiple features.\n",
        "  \n",
        "  In addition to training, this function also prints training progress information,\n",
        "  as well as a plot of the training and validation loss over time.\n",
        "  \n",
        "  Args:\n",
        "    learning_rate: A `float`, the learning rate.\n",
        "    steps: A non-zero `int`, the total number of training steps. A training step\n",
        "      consists of a forward and backward pass using a single batch.\n",
        "    batch_size: A non-zero `int`, the batch size.\n",
        "    training_examples: A `DataFrame` containing one or more columns from\n",
        "      `california_housing_dataframe` to use as input features for training.\n",
        "    training_targets: A `DataFrame` containing exactly one column from\n",
        "      `california_housing_dataframe` to use as target for training.\n",
        "    validation_examples: A `DataFrame` containing one or more columns from\n",
        "      `california_housing_dataframe` to use as input features for validation.\n",
        "    validation_targets: A `DataFrame` containing exactly one column from\n",
        "      `california_housing_dataframe` to use as target for validation.\n",
        "      \n",
        "  Returns:\n",
        "    A `LinearRegressor` object trained on the training data.\n",
        "  \"\"\"\n",
        "\n",
        "  periods = 10\n",
        "  steps_per_period = steps / periods\n",
        "  \n",
        "  # Create a linear regressor object.\n",
        "  my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n",
        "  my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n",
        "  linear_regressor = tf.estimator.LinearRegressor(\n",
        "      feature_columns=construct_feature_columns(training_examples),\n",
        "      optimizer=my_optimizer\n",
        "  )\n",
        "  \n",
        "  # Create input functions.\n",
        "  training_input_fn = lambda: my_input_fn(\n",
        "      training_examples, \n",
        "      training_targets[\"median_house_value\"], \n",
        "      batch_size=batch_size)\n",
        "  predict_training_input_fn = lambda: my_input_fn(\n",
        "      training_examples, \n",
        "      training_targets[\"median_house_value\"], \n",
        "      num_epochs=1, \n",
        "      shuffle=False)\n",
        "  predict_validation_input_fn = lambda: my_input_fn(\n",
        "      validation_examples, validation_targets[\"median_house_value\"], \n",
        "      num_epochs=1, \n",
        "      shuffle=False)\n",
        "\n",
        "  # Train the model, but do so inside a loop so that we can periodically assess\n",
        "  # loss metrics.\n",
        "  print(\"Training model...\")\n",
        "  print(\"RMSE (on training data):\")\n",
        "  training_rmse = []\n",
        "  validation_rmse = []\n",
        "  for period in range (0, periods):\n",
        "    # Train the model, starting from the prior state.\n",
        "    linear_regressor.train(\n",
        "        input_fn=training_input_fn,\n",
        "        steps=steps_per_period,\n",
        "    )\n",
        "    # Take a break and compute predictions.\n",
        "    training_predictions = linear_regressor.predict(input_fn=predict_training_input_fn)\n",
        "    training_predictions = np.array([item['predictions'][0] for item in training_predictions])\n",
        "    \n",
        "    validation_predictions = linear_regressor.predict(input_fn=predict_validation_input_fn)\n",
        "    validation_predictions = np.array([item['predictions'][0] for item in validation_predictions])\n",
        "    \n",
        "    \n",
        "    # Compute training and validation loss.\n",
        "    training_root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(training_predictions, training_targets))\n",
        "    validation_root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(validation_predictions, validation_targets))\n",
        "    # Occasionally print the current loss.\n",
        "    print(\"  period %02d : %0.2f\" % (period, training_root_mean_squared_error))\n",
        "    # Add the loss metrics from this period to our list.\n",
        "    training_rmse.append(training_root_mean_squared_error)\n",
        "    validation_rmse.append(validation_root_mean_squared_error)\n",
        "  print(\"Model training finished.\")\n",
        "\n",
        "  # Output a graph of loss metrics over periods.\n",
        "  plt.ylabel(\"RMSE\")\n",
        "  plt.xlabel(\"Periods\")\n",
        "  plt.title(\"Root Mean Squared Error vs. Periods\")\n",
        "  plt.tight_layout()\n",
        "  plt.plot(training_rmse, label=\"training\")\n",
        "  plt.plot(validation_rmse, label=\"validation\")\n",
        "  plt.legend()\n",
        "\n",
        "  return linear_regressor"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "i1imhjFzbWwt",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "linear_regressor = train_model(\n",
        "    learning_rate=0.00003,\n",
        "    steps=500,\n",
        "    batch_size=5,\n",
        "    training_examples=training_examples,\n",
        "    training_targets=training_targets,\n",
        "    validation_examples=validation_examples,\n",
        "    validation_targets=validation_targets)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "65sin-E5NmHN",
        "colab_type": "text"
      },
      "source": [
        " ## T\u00e2che\u00a05\u00a0: Effectuer une \u00e9valuation sur les donn\u00e9es de test\n",
        "\n",
        "**Dans la cellule ci-dessous, chargez l'ensemble de donn\u00e9es de test et \u00e9valuez votre mod\u00e8le sur celui-ci.**\n",
        "\n",
        "Les donn\u00e9es de validation ayant fait l'objet de nombreuses it\u00e9rations, assurez-vous qu'il n'y a pas eu de surapprentissage des particularit\u00e9s de cet \u00e9chantillon.\n",
        "\n",
        "Les donn\u00e9es de test se trouvent [ici](https://download.mlcc.google.com/mledu-datasets/california_housing_test.csv).\n",
        "\n",
        "Comment vos performances de test se comparent-elles aux performances de validation\u00a0? Qu'est-ce que cela nous apprend sur les performances de g\u00e9n\u00e9ralisation de votre mod\u00e8le\u00a0?"
      ]
    },
    {
      "metadata": {
        "id": "icEJIl5Vp51r",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "california_housing_test_data = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_test.csv\", sep=\",\")\n",
        "#\n",
        "# YOUR CODE HERE\n",
        "#"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yTghc_5HkJDW",
        "colab_type": "text"
      },
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher la solution."
      ]
    },
    {
      "metadata": {
        "id": "_xSYTarykO8U",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "california_housing_test_data = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_test.csv\", sep=\",\")\n",
        "\n",
        "test_examples = preprocess_features(california_housing_test_data)\n",
        "test_targets = preprocess_targets(california_housing_test_data)\n",
        "\n",
        "predict_test_input_fn = lambda: my_input_fn(\n",
        "      test_examples, \n",
        "      test_targets[\"median_house_value\"], \n",
        "      num_epochs=1, \n",
        "      shuffle=False)\n",
        "\n",
        "test_predictions = linear_regressor.predict(input_fn=predict_test_input_fn)\n",
        "test_predictions = np.array([item['predictions'][0] for item in test_predictions])\n",
        "\n",
        "root_mean_squared_error = math.sqrt(\n",
        "    metrics.mean_squared_error(test_predictions, test_targets))\n",
        "\n",
        "print(\"Final RMSE (on test data): %0.2f\" % root_mean_squared_error)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    }
  ]
}
